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Abstract 

We consider maps fx{v) = min^gg^ Aw and gx{v) = ma-x.A^xAv, where 3<C is a 
finite set of nonnegative matrices and by "min" and "max" we mean component- wise 
minimum and maximum. We transfer known results about properties of gx to 
In particular we show existence of nonnegative generalized eigenvectors of fx, give 
necessary and sufficient conditions for existence of strictly positive eigenvector of fx, 
study dynamics of fx on the positive cone. We show the existence and construct 
matrices A and B, possibly not in %, such that f^{v) ~ A^v and g^{v) ~ B'^v for 
any strictly positive vector v. 

1 Introduction 

The theory of nonnegative matrices has been very well developed since its appearance in 
papers of Perron and Frobenius. Vast number of applications to dynamic programming, 
probability theory, numerical analysis, mathematical economics, fractal geometry raise even 
greater interest to this field. As a result there are many strong generalizations of the Perron- 
Frobenius theory (see 0, |, |, 0, |, [T|, |15|, 

The classical Perron-Frobenius theorem shows that a nonnegative matrix has a nonneg- 
ative eigenvector associated with its spectral radius, and if the matrix is irreducible then 
this nonnegative eigenvector can be chosen strictly positive. One important generalization 
of this result was obtained by U.G. Rothblum [0, who studied the structure of the al- 



gebraic eigenspaces of nonnegative matrices and described the combinatorics that stands 
behind the index of the spectral radius and dimensions of the algebraic eigenspaces. More- 
over, it was shown that the algebraic eigenspace of a nonnegative matrix corresponding to 
its spectral radius is spanned by a set of nonnegative generalized eigenvectors with certain 
strictly positive entries. 
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Many generalizations of the Perron-Frobenius theory involve homogeneous monotone 
functions, especially functions of the form 



Qxix) = max Ax, 

where % is a set of square nonnegative matrices of fixed dimension and by "max" we mean 
component-wise maximum. Such functions appear in many well-known problems, such as 
the theory of controlled Markov chains, Leontief substitution systems, controlled branching 
processes, parallel computations, transportation networks, etc. The study of maps goc was 
initiated by Richard Bellman. Using the Brouwer fixed point theorem he proved existence 
of a strictly positive eigenvector of the map gx in the case when each matrix in % is 
positive and studied the asymptotic behavior of iterations g^i'^) = 9oc{goc{- ■ ■ gx{v) . . .)) 
for a nonnegative vector v (see @ and |P, chapter XI, sections 10 — 11]). These results were 
generalized to a set of irreducible matrices by P. Mandl and E. Seneta |I4| . 

The most important results for our investigation were obtained by W.H.M. Zijm in 



| 25| . He showed that there is a simultaneous block-triangular decomposition of the set 
of matrices %, which was used to give the necessary and sufficient conditions for the 
existence of a strictly positive eigenvector of gx and extend the above mentioned result of 
U.G. Rothblum on nonnegative generalized eigenvectors to gx. Related results can be found 
in [^, Chapter 35] . Independently, Karel Sladky ^ obtained the same block-triangular 



decomposition and used it to get bounds on the asymptotic behavior of iterations fi'x(^) 
for a nonnegative vector v. Stronger results about asymptotic behavior of iterations g^i'^) 



were obtained in |22, p3l Eq] for the case when some special matrices in % are aperiodic. 



We consider maps of similar form, but with "minimum" instead of "maximum" 

ficix) = miny4x. 



These maps appear in |12] in connection with the construction of "self-similar" metrics 



on self-similar sets and finding their Hausdorff dimensions. Also such maps appear in the 



study of growth of Schreier graphs of groups generated by finite automata [TT|. These 
problems demand us to study spectral properties of maps jx and describe asymptotic 
behavior of its iterations. 

Considering maps fx and gx we can always suppose that the set % satisfies the product 
property, i.e. % is constructed by all possible interchanges of corresponding rows selected 
from a finite set of square nonnegative matrices (see the precise definition and explanation 
in Section ^). Under this assumption, for every vector v there exist A = ^ % and 
B = E % such that fx{v) = Av and gx{v) = Bv. In the theory of Markov decision 
processes this property is usually called the optimal choice property (see [P, §). 

The asymptotic behavior of iterations f^{v) is studied with respect to the following 
equivalence. Let a„, bn, n > 1, be sequences of nonnegative numbers or vectors of the same 
dimension. We say that ^ 6„ if there exists constant q > such that an < q ■ bn for 
all n large enough. If a„ ^ bn and 6„ ^ cin then we say that a„ ~ 6„ and that a„ and 6„ 
have the same growth. Then h"'{v) ~ h"'{u) for any homogenous nondecreasing function 
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h : M.^ and any strictly positive vectors v, u. Hence we can and we will change one 

strictly positive vector to another one considering asymptotic behavior if it is necessary. 

Considering maps f% we follow as close as possible to the ideas of W.H.M. Zijm and 
use his paper as a model. Notice that we cannot use Zijm's results for —fx, which 
can be expressed using maximum, because matrices should be nonnegative and dynamics 
is considered on the nonnegative cone. The problem in transferring the results obtained for 
9% to foe lies in the convexity property which fjc lacks. In particular, there is no simulta- 



neous block-triangular decomposition, which was extremely important in ||2^, ^ To 
overcome this difficulty we show that if the set % satisfies the product property then there 
exist matrices B and C in IK which give the lowest and the greatest asymptotic behavior 
over all matrices in %, i.e. B^-v ^ A^v ^ C"'v for all A ^ % and every strictly positive vec- 
tor V. These matrices we call respectively :<-minimal and ■<-maximal for the set %. Using 
these notions we study spectral properties of foe- In particular, we prove that foe possesses 
a strictly positive eigenvector if and only if some (every) ^-minimal matrix possesses a 
strictly positive eigenvector. The main result shows existence of nonnegative generalized 
eigenvectors of /x- Finally as a corollary we describe the asymptotic behavior of each com- 
ponent of f^{v) by showing that f^{v) ~ A^v for some (every) ^-minimal matrix A. We 
prove some other propositions similar to the results of |2^, sometimes assuming additional 
conditions. 

I would like to thank Stephane Gaubert for bringing my attention to Zijm's articles 
and Volodymyr Nekrashevych for helpful suggestions. 



2 Nonnegative matrices: definitions, notations, results 

We recall in this section some (usually well-known) definitions and results that we need 
about nonnegative matrices (for the references see 0, Chapter 2], [|T], Chapter 1], 
Chapter XIII]). 

A matrix A = {aij) is called nonnegative {positive) if a^j > {a.ij > 0) for all indices 
Denote by Ai the ith row of the matrix A and by Vi the zth component of a vector v. 
A vector v is called strictly positive if f j > for all i. Unless otherwise stated, all matrices 



will be square of a fixed dimension A^. Following ||^, 0, ^ the set {1,2,..., A^} is called 
the state space and denoted by S. If Si, S2 C S then we denote by A\(^Si,S2) the restriction 
of the square matrix A to Si x S2 and by v\si the restriction of the vector v to Si. 
The spectral radius of a matrix A is denoted by spr{A). 

We say that state i has access to state j if there exists a nonnegative integer n such 
that the ijth entry of A^ is positive. Matrix A is called irreducible if any two states have 
access to each other. In the other case A is called reducible. 

The following theorem states some important properties of square nonnegative matrices. 

Theorem 1 ([^, Chapter 2]). Let A be a nonnegative matrix with spectral radius A. Then 
a) A is an eigenvalue of A. 
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b) There exists a nonnegative eigenvector v associated with A. 



c) If Au > au for u > then X > a. 
If moreover A is irreducible then 

d) There exists a strictly positive eigenvector v associated with A and any nonnegative 
eigenvector of A is a scalar multiple of v. 

e) If Au > Xu or Au < Xu for u > then Au = Xu. 

f) {al - Ay^ > for any a > X. 

g) spr{A\(^c,c)) < ^ for any C S. If A is reducible then spr{A\(^c,c)) < ^ for any 
C <^ S and spr{A\(^c,c)) = ^ for some C ^ S. 

h) If Au > au (Au < au) for u>0 then X> a (respectively, X < a). 

Iterations of a matrix heavily depend on its block-triangular structure. We will describe 
this following Q, ^ . 

A class of a nonnegative matrix A is a subset C of the state space S such that ^41 (c^c") is 
irreducible and such that C cannot be enlarged without destroying the irreducibility. A class 
C is called basic if spr{A\(^c,c)) = spr{A), otherwise nonbasic (when spr{A\(^c,c)) < spr{A)). 
It follows that for any matrix A we have a partition of the state space S into classes, say 
Ci, C2, . . . , C„. Then, after possibly permuting the states and renumbering the classes, A 
can be written in the form, sometimes called the Frobenius Normal Form, 



A 



where denotes A\(^Ci,Cj)- Hence classes can be partially ordered by accessibility rela- 
tion. We say that a class C has access to (from) a class C if there is an access to (from) 
some (or equivalently any) state in C to some (or equivalently any) state in C. A class is 
called final if it has no access to any other class. 

The spectral radius of a class C is the spectral radius of A\(^c,c)- 

The next proposition describes when a matrix A has a strictly positive eigenvector and, 
what is more important for the subject of this paper, when {A^v)i ~ {A^v)j for all indices 
i,j and any strictly positive vector v. 

Proposition 2. Let A be a nonnegative matrix with spectral radius X. Then the following 
conditions are equivalent: 

a) The matrix A has a strictly positive eigenvector. 

b) The basic classes of A are precisely its final classes. 





^(1,2) 


••• ^(l,n)^ 





^(2,2) 


• • • ^(2,n) 










V 
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c) {A'^'v)i ~ A" for all i and for some (every) vector v > 0. 

d) {A^v)i ~ {A^v)j for all i,j and for some (every) vector f > 0. 

Proof. The proof of equivalence a) and b) can be found in 0, Theorem 3.10]. The proof of 
the rest will follow directly from Corollary ^ □ 

Also notice that if a nonnegative matrix possesses a strictly positive eigenvector then 
it is associated with the spectral radius of this matrix. 

Already the last proposition indicates importance of the position of basic and nonbasic 
classes of a square nonnegative matrix A for existence of a strictly positive eigenvector and 
behavior of its iterations. These positions can be defined precisely by introducing the con- 
cept of a chain. A chain of classes of A is an ordered collection of classes {Ci, C2, . . . , C„} 
such that Ci has access to Cj+i, i = 1, . . . ,n — 1. The length of a chain is the number of 
basic classes it contains. The depth of a class C of A is the length of the longest chain that 
starts with C. The degree i^iA) of A is the length of its longest chain. Let Si be the union 
of all classes of depth i. The partition {S'o, 5*1, . . . , 5*,^} of the state space S is called the 
principal partition of S with respect to A. Principal partitions play a fundamental role in 
this paper. The next result is then straight forward. 

Proposition 3. Let {Sq, Si, ... , S^} be the principal partition of S with respect to A. Then, 
after possibly permuting the states, A can be written in the form 

■ ■ ^(1^-1,0) 

^(0,0) / 

where ^(jj) denotes A\(^Si,Sj)- We have that spr(74(o,o)) < spr{A) (if Sq is not empty); 
spr(A(j^j)) = spr{A) and the final classes and basic classes of A(^i^i) coincide fori = 1, . . . ,1/. 
Each state in S'j+i has access to some state in Si for i > 1 (here v4(j^j_i) is non-zero). 

Notice that it follows from Proposition |^ that A\(^Si,Si) possesses a strictly positive 
eigenvector for every i = 1, . . . ,1/. 

We need the following useful lemma. 

Lemma 1 ([^, Lemma 2.5]). Let A be a nonnegative matrix with spectral radius A. 

a) If Av > av for some real number a and a real vector v with at least one positive 
component, then X > a. 

b) If Av > Xv with V > then every final class of A is basic and {Av)i = Xvi for every 
i in a final class of A. 

Matrices which possess strictly positive eigenvectors have the following additional prop- 
erties. 



A 



A 



{u,u) 



A 



A 






{u,u-l) 
{u-l,u-l) 
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Lemma 2. Let A he a nonnegative matrix with spectral radius A which has a strictly 
positive eigenvector. Let Si G S be the union of all final classes of A. If Au = Xu with 
u\si > then w > 0. 

Proof. Let S2 = S\Si. Then, after possibly permuting the states, A can be written in the 
form: 

Each class C in 5*2 has access to some state in Si, spr{A\(^c,c)) < by Proposition 0, 

and (A/ — A\{^c,c)) ^ > by Theorem |l| item f). It follows that (A/ — ^|(s'2,5'2)) ^ ^{S2,Si) 
has a positive element in each row. Then 

^Isa = (A/ - ^|(52,S2)) ^^I(52,5i)m|5i > 0. 

□ 

Lemma 3 (P5|, Lemma 2.3]). Let A he a nonnegative matrix with spectral radius A which 
possesses a strictly positive eigenvector. Then: 

a) There exists a nonnegative matrix A* defined by: 



A* = lim 

n-*oo n + 



n 

+ 1 ^ 

i=0 



We have AA* = A* A = \A* and (A*)^ = A*. Moreover, a*j > if and only if j 



belong to a basic class of A and i has access to j under A. 

b ) The matrix XI — A + A* is nonsingular. 

c) If A*v = for some vector v >0 (or v <0), then Vi = for every state i belonging 
to a basic class of A. 

d) If Av > Xv (or Av < Xv) for some vector v then A*v > v (respectively, A*v < v). 

Notice that if A is a (reducible) stochastic matrix then A* is a limiting transition 
probability matrix and the inverse of [I — A + A*) is the so-called fundamental matrix of 
the respective Markov chain. 

Asymptotic behavior of iterations of a nonnegative matrix can be studied through 
its generalized eigenvectors corresponding to its spectral radius. Let ^4 be a nonnegative 
matrix with spectral radius A. The index rj{A) of A with respect to A is the smallest 
integer n such that the null spaces of {A — A/)" and {A — A/)"^^ coincide. The elements 
of Null(v4 — Xiy \ Nu11(j4 — A/)*~^ are called the generalized eigenvectors of order i. It was 
proved in |[T^, Theorem 3.1] that r]{A) = iy{A). Moreover, it was shown that generalized 
eigenvectors can be chosen nonnegative with special strictly positive components. More 
precisely (see also |jl9|, p5|): 
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Theorem 4 (|]T^ Theorem 3.1]). Let A be a nonnegative matrix with spectral radius A. 
Let {Sq, Si, ... , S^} be the principal partition of S with respect to A. Then there exists a 
set of nonnegative generalized eigenvectors v^^\ v^'^\ . . . , v^'^^ such that 

Av'^''^ = Xv^^\ 

Av^'^ = Xv^'^+v^'+^\ z = z/- 1,...,2,1. 

Moreover 

V i— 1 

^ > 0, 3^[jSk and ^ = 0, j G |J S^. 

k=i k=Q 

To give estimates on the growth of A^v we need the following lemma. 
Lemma 4. For any integer k > and real X, (3 > we have asymptotic relation 



Proof. The asymptotic relation '^IZq i'^ ~ n^+^ is standard. Then for X = (3: 

n—l n—1 



i=0 i=0 

and for /9 > A we have inequalities: 

n— 1 n—l / /3\ * //^\* 

i=0 1=0 ^ ^ i=0 ^ ^ 



(iY - 1 

< X^'n'^^l <n'' ^ 



i -^ - £ _ 1 ' 

A A 



□ 



Corollary 5. Let A be a nonnegative matrix with spectral radius X. Let {Sq, Si, ... , S^} 
be the principal partition of S with respect to A. Then 

~ n^-^A", for keS^, i>l 

for any strictly positive vector v. 

Proof. The statement follows from a general result about asymptotic behavior of matrix 
powers obtained in [jl8|. We sketch the proof to use it later. 

Using the identities for the generalized eigenvectors f from Theorem ^ and the above 
lemma, one can get by induction that 

~ A" ^ n^v'^'+^\ for i = I/, . . . , 1 (A"^;^^))^ ~ n^^U", for k G Si. 

j=0 

Since spr{A\(^So,So)) < A the S'oth components of a strictly positive vector v does not effect 
the asymptotic behavior of A^vlsxso- Hence A^vlsxso ~ ^"^'•^^|s\s'o- ^ 
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Remark. Corollary ^ gives us an algorithm of finding the growth of each component 
of A^v (see the detailed analysis in [l^). For indices in Si for z > 1 it follows directly 
from the corollary. For z G 5*0 we consider the matrix A\(^So,So) ^md its principal partition 
and so on. This algorithm can be also described using chains of classes as follows. Take a 
state i and the corresponding class Ci which contains i. Let jS be the maximum of spectral 
radii of classes C, where C runs through all classes such that Ci has access to C. Consider 
all possible chains that start at Cj and for each chain count the number of classes C 
in this chain with spectral radius /3. Let k be the maximal among such numbers. Then 
{A'^v)i ~ n^~^(3^ for any strictly positive vector v. In particular, if i belongs to a final class 
of A then {A^v)i ~ where 13 is the spectral radius of the class. 

Corollary |^ implies that the components of A^v are comparable with respect to the 
partial order ^. The ^-minimal possible growth of {A^v)i over all indices i is ~ 7", 
where 7 is the spectral radius of some final class C. If state i has access to state j then 
{A^v)j ^ {A^v)i. So, {A^v)i ~ {A^v)j for any two states i and j in the same class of A. 

3 Existence of strictly positive eigenvector of f% 

Let % he a, finite set of nonnegative matrices. In this section we define the notion of a 
^-minimal matrix and the principal ^-minimal partition of S with respect to %. Using 
these notions we give necessary and sufficient conditions for existence of a strictly positive 
eigenvector for the map /ac(t') = min^g^^'^^- 

Note that in general we do not have the property that for every vector v there exists 
A & % such that f-xiv) = Av. The following concept eliminates this difficulty (compare 
with ^ Section 3.1]). 

Definition 1. Let 3C be a set of nonnegative N x N matrices. We say that % satisfies 
the product property if for each subset V S and for each pair of matrices A,B & % the 
matrix C defined by 

^ / A, iiieV; 
^ ■ \ B,, iii e S\V. 

belongs to %. 

If we have any finite set %q of nonnegative matrices then we can close it with respect 
to the product property and obtain another finite set %. We just take all possible matrices 
C obtained as follows: the ith row of C is the ith row of some matrix from %. Then it is 
easy to see that 

min Av = min Av 

for any vector v. So we can extend our given set of matrices to a bigger one, which satisfies 
the product property, without changing the map fx. Moreover, for every v there exists 
A = Ay e% such that fx{v) = Av. 

Hence we will always assume that % possesses the product property. 



8 



As was mentioned in introduction, Richard Bellman in considered compact sets % of 
positive matrices and proved that gx has a strictly positive eigenvector. It was generalized 



to a set of irreducible matrices in [14]. A simple proof of this result was obtained by 
W.H.M. Zijm p5| following the arguments in |1I0| , Appendix B]. His proof also works for 
the maps foe- 

Proposition 6. Suppose that every matrix in the set % is irreducible. Then fx possesses 
a strictly positive eigenvector associated with Xx = miuA^x spr{A) . Moreover, it is unique 
up to a scalar multiple. 

Proof. Take any B & %. Let As be the spectral radius of B and let v be the corresponding 
strictly positive eigenvector. Find D & % such that 

Dv = minAv 

with A = Bi if {Bv)i < {Av)i for all A e X. If D = B then fx{v) = Bv = Xbv and 
we are done. If D B then Dv < Bv = Xbv and Xd '■= spr{D) < Xb by Theorem |1| 
item e ). Apply the same procedure for the matrix D with its strictly positive eigenvector u 
associated with A^. Since % is finite, after a finite number of steps we will reach a matrix 
M with spectral radius A and eigenvector w such that 

Mw = min Avj = Xvj. 

Since Aw > Xw for every A & %, we get A = min^iggc spr (A) by Theorem |I| item e). 

Let u,v > he eigenvectors of fx and let fx{v) = Av = Xv and fx{u) = Bu = Xu. 
Then spr{A) = spr{B) = A, Bv > fx{v) = Xv, and Bv = Xv by Theorem |l]item e). Hence 
by Theorem |I] item d) the eigenvector of fx associated with A is unique up to a scalar 
multiple. □ 

The following lemma is an important result for understanding the asymptotic behavior 
of fxiy)- It will be used throughout the paper. 

Lemma 5. There exists B E % such that B'"'v ^ A"'v for any A E % and v > 0. 

Proof. We use induction on dimension A^. For N = 1 the statement is obvious. Suppose 
the lemma is correct for any dimension < A^. Let us fix f > 0. 

For each A E % and i E S we can find the asymptotic behavior of {A^v)^ using 
Corollary |^. Define the set %' (Z % of all matrices B in % for which there exists i E S such 
that {B''v)i ^ (v4"f )j for all A E X and j E S. Note that it follows from Corollary | that 
{B^v)i ~ A" for some real A > 0. For each matrix B E%' define 

So{B) = {jeS\ (B^v), ~ iB^v)j ~ A"} 

and Si{B) = S\So{B). Suppose some state i in So{B) has access to some state j in Si{B). 
Then {B^v)j ^ {B"'v)i. Since the asymptotic behavior of {B"'v)i is ^-minimal for B, we 
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have {B^v)i ~ {B"'v)j. Hence j G So{B) and we have a contradiction. Thus no state in 
So{B) has access to any state in Si{B), which means that -B|(So(-B),Si(b)) = 0- 

Observe that the spectral radius of every class of B from So{B) is not greater than A. If 
a class C from So{B) is final then it has spectral radius A. The converse is also true: a class 
C from So{B) with spectral radius A is final. Really, suppose it is not final. Then it has 
access to a final class from So{B). Thus there exists a chain which start at C that contains 
at least two classes with spectral radii A. So, {B^v)i >z nX"' for z in C by Corollary ^. 

Let us show that %' contains a matrix B with the biggest set Sq{B), i.e. such that 
So{B) D So{A) for any A G %'. It is sufficient to show that for any matrices B and D from 
%' there exists E e X' such that So{E) D So{B) U So{D). Define E as follows: Ei = Bi for 
i G So{B) and Ei = Di for i ^ Sq{B). 

E = ( * A 

V B\^So{B),So{B))J 

Then E e X' and S'o(^) D 5o(5), because = for z G 5o(5)- In order to 

prove that So{E) contains So{D), it is sufficient to prove that each class C of E, which 
belong to So{D) \ So{B) with spectral radius A is final (if it is empty we are done). By 
construction E\(^c,c) = D\(c,c) and C belongs to some class C of D from So{D). \i C ^ C 
then spr{D\(^c',c')) > spr{D\(^c,c)) = sp^{E\(^c,c)) = by Theorem |l] item g) and we have 
contradiction with C C <S'o(-D). Thus C = C and E\(^c,s\c) = -D|(c',5\c') = 0. So C is final 
and our claim is proved. 

Choose i? G X' to be a matrix with the biggest set So{B). Denote 5*0 := So{B) and 
^1 := SiiB). 

If 5*0 = then we are done - the matrix B satisfies the condition of the lemma. Suppose 
that Si 7^ 0. The set X\(^Si,Si) satisfies the product property and we can apply induction 
to it. So there exists D & X such that {D\(^Si,Si))^v\si ^ {A\(^Si,Si))"'v\si for any A E X. 
Define a matrix E in the same way as above: E^ = Bi for i E Sq and Ei = Di for i ^ Sq. 
We want to show that it satisfies the condition of the lemma. 

Again {E"'v)i = {B"v)i for i G ^o- So {E''v)i ^ (A"^;)^ for any matrix A e X for i e Sq. 
We need to prove the previous inequality for i E Si. 

n—1 
1=1 

n—1 n—1 

1=1 1=0 

Fix i in Si and let {{D\(^Si,Si))"'v\si)i ~ n^/?". Suppose < A. Then there exists j G Si 
such that {iD\^s„s,)Tv\s,)j ~ Then: 

n—1 n—1 
1=0 1=0 
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and therefore j must be in Sq- We get a contradiction, hence (3 > \. 
If /? > A then 

n— 1 n—l 
l=Q 1=0 

for every A 

Now suppose that /3 = A. Let Cj be the class of D\i^Si,Si) that contains i. Then 
A is the maximum of spectral radii of D\(^Si,Si)\{c,c)^ where C runs through all classes 
of D\i^Si,Si) such that Cj has access to C. Also the maximal number of classes C with 
spr{D\(^Si,Si)\{c,c)) = A in chains that start at Cj is k. If the maximum of spectral radii 
of D\i^c,c)i where Cj has access to C, is greater than A, then {D^v)i >z n^^^X"'. If not 
then the maximal number of classes C oi B with spr{D\(^c,c)) = A in a chain that starts 
at Ci is at least + 1, otherwise there exists a state j in Si with {E"'v)j ~ A". Thus, 
{D"'v)i >z ri'^+^A". Notice that the above statement is true for any matrix ^4 G IK, i.e., if 
iiA\^Si,Si)Tv\s,)i ~ ri'^A" then (A"^;)^ h n^+U". Then 

n—l n—l 
1=0 1=0 

for any A e X. So {E''v)i ^ {A'^v)i for all t and A e X. □ 

By similar arguments one can show that there exists C G 3C such that A"'v ^ C"f for 
every A & X. 

Definition 2. A matrix B E X which satisfies Lemma ^ will be called :<-minimal for X. 

There is a simple (but not effective) algorithm to find all ^-minimal matrices. We find 
the asymptotic behavior of {A^v)i for all A G 3C by Corollary |^ and take matrices with 
^-minimal growth (such matrices exist by Lemma 1). 

If the spectral radius of a matrix A is zero, then A is nilpotent and asymptotic behavior 
of A^v is trivial. Consideration of such matrices is elementary but does not fit precisely 
in the discussion below. To avoid these unnecessary complications and without loss of 
generality in the sequel all considered matrices have spectral radius > 0. 

The principal partitions, spectral radii and degrees of every two ^-minimal matrices 
coincide, which follows from Corollary ^ and from the fact that A^v ~ B'^v for any ^- 
minimal matrices A^B eX. Notice that the spectral radius of a ^-minimal matrix is equal 
to mm A^-xspr {A). Denote A = spr{B) and ly = i'{B) for a ^-minimal matrix B EX. 

Definition 3. The principal partition {S'o, iSi, . . . , S^} of a ^-minimal matrix is called the 
principal ^-minimal partition of S with respect to X. 
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The following proposition gives a sufficient condition for existence of a strictly positive 
eigenvector for the map fx. Moreover, it will follow from Corollary |12| that this condition is 
also necessary, and what is more important for the subject of this paper that it is equivalent 
to the property that all components of the iterations /x(^) have the same growth. 

Proposition 7. Suppose that some :<-minimal matrix possesses a strictly positive eigen- 
vector. Then fx possesses a strictly positive eigenvector associated with A. 

Proof. Note that by Proposition |^ if one ^-minimal matrix possesses a strictly positive 
eigenvector then all ^-minimal matrices do. 

Let I? be a ^-minimal matrix with strictly positive eigenvector v. Apply the same 
procedure as in the proof of Proposition ||. Find D e IK such that 

Dv = min Av 

Ag3C 

with A = Bi if {Bv)i < {Av)i for all A e X. Then Dv < Xv and D'^v < X^v = B'^v. Thus 
D is ^-minimal, has strictly positive eigenvector and spr{D) = A. Since each final class of 
D is basic, {Dv)i = {Xv)i for all i in final classes by Theorem |I| item e). Hence Di = Bi for 
i in the final classes of D and the set of final classes of B contains the set of final classes 
of D. 

By Proposition Q each nonfinal class of D is nonbasic. Let Si <Z S he the union of all 
final classes and let S2 = S \ Si. Then, after possibly permuting the states 



D 



D\{S2,S2) E 



\{Si,Si) 



B\ 

with spr[D\(^Si,Si)) = ^ and spr{D\(^s2,S2)) < ^- Define 

u\si = v\si and u\s2 = i>^I - D\(^s2,S2)y^ Ev\s2- (1) 

Then Du = Xu and thus -u > by Lemma ^ Suppose Ui > Vi for some i E S. Then it 
follows from Du = Xu and Dv < Xv that 

D\{S2,S2) u\s2 - v\s2 > A u\s^ - v\s2 

This contradicts spr{D\(^s2,S2)) < by Lemma |l]. Hence > m > 0. 

By construction u = v if and only if D = B. We can apply the same procedure to D 
and u. On each step the set of final classes of the new matrix is contained in the set of 
final classes of the previous matrix and the next eigenvector coincides with the previous 
one on the states from final classes of the new matrix. Since % is finite, after some steps all 
received matrices will have the same set of final classes and all received eigenvectors are the 
same on this set. Now suppose this process will never stabilize. It means that all received 
eigenvectors are different. Since % is finite, some matrix appears in this process at least 
two times with different strictly positive eigenvectors that coincide on the final classes of 
this matrix. But by ([I|) eigenvector of a matrix is uniquely defined by its coordinates from 
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the final classes of this matrix. We get a contradiction. Thus after a finite number of steps 
we will reach a ^-minimal matrix M G X with strictly positive eigenvector w such that 

Mw = rain Aw = Xw. 

□ 

Corollary 8. Under the conditions of Propositions^ or ^ the asymptotic relation 

holds for any strictly positive vector u and i E S. 

So, if there exists a ^-minimal matrix with strictly positive eigenvector, then the growth 
exponent of each component of f^{v) is equal to the spectral radius of this ^-minimal 
matrix. 

The next proposition with p = 1 gives the basis of induction for Lemma |^. 

Proposition 9. Let {Sq, Si, ... , S^y} be the principal ^-minimal partition of the state space 
S with respect to %. Then there exists a nonnegative vector w with w\s^ > such that 

min Aw = Xw. 

Proof. Let B be any ^-minimal matrix. Since {Sq, Si, . . . , S,^} is the principal partition of 
B, the matrix B\^s^^s^) possesses a strictly positive eigenvector v associated with A. The 
set %\s^ = {^1(5^, s^); ^ ^ ^} also satisfies the product property and B\s^ is ^-minimal for 
it. We can apply Proposition |^ for "Xls,,- There exists a strictly positive vector u defined 
on Si, such that 

minyllfs s" 1^ = Xu. 

Take w such that w\s^ = u a.iadw\s\s^ = 0. Then w satisfies the condition of the proposition. 

□ 



It was shown in |25] and |22| that a stronger result holds for gx, which proves existence 
of a simultaneous block-triangular representation of the matrices in % and allows one to 
define the "principal partition" of S with respect to %. This partition plays a fundamental 
role in those papers. This result doesn't hold for fx. 

4 Generalized eigenvectors of fx 

We prove in this section two lemmata from which the main result follows immediately. The 
first lemma proves existence of a solution of a set of "nested" functional equations. As it 



was noticed in [g5|, it can be viewed as a generalization of the Howard's policy iteration 
procedure 

Let t be an integer greater than 1. Suppose that for each A E % we have a sequence of 
vectors ri{A), i = 1, . . . ,t — 1. 
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Lemma 6. Assume that the set of rectangular matrices 



{{A,r^{A),r2{A),...,r,.,iA))\AeX} 

satisfies the product property. Suppose that there exists (for all) a :<-minimal matrix B E % 
with a strictly positive eigenvector v. Suppose furthermore B*rt-i{B) > for any ^- 
minimal matrix B (here B* is defined in Lemma^. Then there exists a solution {v^^\ . . . , v^^'^} 
of the set of functional equations: 

mill = Av^*^ 

mm {Av'^'-^^ +ri_ A A)] = Xv^'-^^ + v^^ , i = 2,...,t, 

where %i is defined recursively by 

Xt := {A \ AeX,Av^'^ = Xv'^'^, 

X, := {A\ AeX,+i,Av^''> + ri{A) = Xv^^ +v^'+^^, i = 2,...,t-l. 
Furthermore v^*^ > 0. 
Proof. The set of equations 

= Xv^''^ + v^'-^'\ i = l,...,t-l, (2) 

B*v^^^ = 

has a unique solution 

= {XI -B + B*)-'[ri{B) + B*r^^i{B)-v^'+% t = 2,...,t-l, 
= {XI - B + B*)-'[ri{B) -v^% 

Moreover v^^^ > 0. Since we have the "extended" product property, there exists a matrix 
D eX such that 

Dv^'^^ = minAf(*\ 
Dv^^+rJD) = min {Av^^+ri{A)],i = l,...,t-l, 

where "Ki G X denotes the set of matrices which minimize the right hand side of zth 
equation above. We choose D = B if B satisfies above equations, i.e. if S G JCi. 

Then Dv^^"^ < Xv^^'^ and thus D is ^-minimal and possesses a strictly positive eigen- 
vector. As above, the set of equations (0) with the matrix D instead of B has a unique 
solution {u^^\ . . . ,u^*^ with u^*^ > and so on. We want to show that this process will 
eventually stop. It is easy to see that if v^^^ and -u'-*-' satisfy the following properties 
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(a) < vW; 

(b) if ti^*^ = foT i = k + 1, . . . ,t then -u^^^ < t;^''); 

(c) if u^'^ = v^'^ for alH = 1, . . . , t then D = B, 

then, since % is finite, after a finite number of steps we will reach a matrix which stays 
intact under application of this process. The corresponding solution of (|) will satisfy the 
conditions of the lemma. 

Let us prove (a), (b) and (c). Let C C be the union of all final classes of D. 

(a) Using Lemma ^ and construction of u^^^ and v^^"^ several times we get 

= = D*[Du^''^^ - An(*-i) + ri_i(D)] = D*rt-i{D) < 

(b) Now suppose u^^^ < v^^^ ioi i = k + 1, . . . ,t. Define vectors tp^^\ i = 1, . . . , t, such 
that: 

= At;W+^«, 
Dv^^+ri{D) = \v^^ + v^'+^^ + 

From d^) for the matrix D and the previous equations we get: 

- tiW] = A[t;« - M«] + - u^'+^^] + (3) 

Thus, = and Dv^'^ + ri{D) = Bv^'^ + n^B) ioi i = k + I, . . . ,t. Hence B G "Kk+i- It 
follows that if)^'^'^ < and 

D[v^^) - = \[v^^) - u^^)] + ^C^) ^ ( applying D* ) (4) 

= 0. 

Hence ipf^"^ = for i G C by Lemma ^ item c j. 

Consider the case k > 2. Then ?/;f < for i G C and hence D*^)^^-^^ < by 
Lemma ^item a). Applying D* to (A; — l)st equation of we obtain: 

= D*\v^''^ - M^'^)] + D*^^^-^\ but - m^'^)] < X\v^'''^ - u^''\ 

Hence - m^'^)] > - u^^'^] = -D*^Ij^''-^^ > 0, because V'f < for z G C. 

For A; = 1 we have B G and since ip^^^ = for i & C we may choose = Bi for 
z G C. In this case D* = B* and ^ = ^ = for z G C. Thus D*^;^^) = 0. It follows from 
(I) that 

[^(1) [^(1)_„(1)]=0. 

(c) As above, ■?/''■*•' = for all i and hence B G !Ki. Thus D = B hj construction. □ 
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Lemma 7. Let {So, Si, . . . , S,y} be the principal :<-minimal partition with respect to %. 
There exists a set of nonnegative vectors v^^\v^'^\ . . . ,v^'^^ such that 

minAv^"') = \v^''\ (5) 



mm Av^^ = Xv^'^ + v^'^'^\ i = u - 1, . . . ,2,1 



where 



X, := {A \ AeX,Av^''^ = Xv^'^, 

Xi := {A\ AeXi+i,Av^'^ = Xv^'^ +v^'+^^, i = 2,...,v-\. 



Moreover 



> 0, J G U ^fc ^ = 0, J G U S^. (6) 

k=i k=0 

Proof. By induction on u. For u = 1 the result follows from Proposition ^ Suppose that 
the lemma holds for u < t and let now v = t. 
Notice that 

Xt = {A \ AeX, Av^^ = Xv^'^ and A| (5^5^,5,) = 0} 
for any given f*^*^ such that t'^*''|5\5t = 0. Define the set of matrices 

:K = {A\^s\s„s\s,),AeXt}. 

Clearly J{ also satisfies the product property and B\(^s\s^,s\s^) is a ^-minimal matrix for J{ 
for any ^-minimal matrix B for X. Thus Sq, Si, . . . , Si,-i is the principal ^-minimal parti- 
tion of IK. By the induction hypothesis there exist nonnegative vectors u^^\ u^'^\ . . . , ti^*"^) 
defined on S\Sjj such that Mj > for i G St-i and 

minylu^*-^) = Xu^'-^\ 

min Au^''> = Xu^''> + u^'+'^\ i = t - 2, . . . ,2,1. 

Now we need to find vectors v^^\v^'^\ . . . , v^*^ such that (|^) holds. Let us take 

f = u^j^ and t'j*'' =0 for j G \ S"*. 

Then Xi C {A \ A E Xt, A\(^s\St,s\St) ^ for 2 = — 1, and the vectors v^''\ 

independent of their coordinates on St, satisfy (||) for states in S\St. It remains to determine 
f]*'' for j E St, i = 1, . ■ . ,t. The conditions on v^^^\st are the following: 

AG Jv 

min \A\^SuS,)V^'^U + Yl'^\iSuS,)W^'^\s} = Xv^'^U + v^'^'^\s,, i = t - 1, . . . ,2,1. 



16 



Since {Sq, Si, ... , St} is the principal partition of any ^-minimal matrix B ^ %, the matrix 
B\{St,St) possesses a strictly positive eigenvector associated with A. Moreover u^^~^^\st-i > 0. 
Each final class of B\(_St,St) has access to some state in B\(^St-i,St-i)- Thus 

for some i in every final class of B\(^St,St)- Then B\*^g^ g^-^B\(^St,St-i)''^^^^^^\st-i > 
^-minimal B by Lemma ^ item a). We can now apply Lemma ^ and find v^^^ls^. 

It may happened that v^^'^ does not satisfy the nonnegativity constrains ® on St (they 
satisfy it on S* \ St by induction). In this case consider 

= (7) 
= + z = l,...,t-l. 

They also satisfy (|^) and we can choose a large enough so that Wj'^ > for all j G 5'^, 
i = l,...,t. □ 

Now we are ready to prove the main result. 

Theorem 10. Let {Sq, Si, ... , S,,} be the principal :<-minimal partition with respect to %. 
Then there exists a set of nonnegative vectors v^^\v^'^\ . . . ,v^'^^ such that 

AgOC 



minAv^^ = Xv^^ + v^'+'^\ i = u - 1, . . . ,2,1. 



Moreover 

V i— 1 



f^>0, jelj^fc and ^;f = 0, j G |J 



k=i k=0 



Proof. Use Lemma |^ to find solutions v^-^\ v^'^\ . . . , v^'^'' of the corresponding system (|^). 
Now consider the vectors w^^\ u)^'^\ . . . , w^'^^ from (0). It is easy to see that for a large 
enough 

min Aw^''^ = min Aw^^'^ = . . . = rain Aw^''\ i = 1, . . . 

Hence for a large enough the vectors w^^\w^'^\ . . . , w^'''^ satisfy the conditions of the theo- 
rem. □ 

Corollary 11. Let {So, Si, . . . , S^} be the principal ^-minimal partition with respect to 
%. Then 

ifoci'v))i ~ n^"^A", where i G Sk, 
for any strictly positive vector v and k > 1. Moreover, for any :<-minimal matrix B E % 

if^v)), ~ (B-v), 

for any strictly positive vector v and i E S. 
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Proof. The proof of the first part is the same as for a single matrix (see Corollary |^) . Thus 
~ {B"-v)i for i ^ Sq. We need to prove this asymptotic relation for i E Sq. 
The upper bound f^{v) ^ B'^v is obvious. Define 

% = {A e X \ A\^So,s\So) = 0} and /xlso = min A|(So,So)- 

Then B\(^So,So) is ^-minimal for Oi\so ioi any matrix B ^-minimal for %. Let /? = spr{B\(^So,So)) 
for ^-minimal B (notice that (3 < X) and let {S'q, S[, . . . , S^,} be the principal ^-minimal 
partition of 5*0 with respect to O-CIsq- By Theorem ^ there exist nonnegative vectors 
w^-^\ . . . , w^'^ ^ defined on such that 

fxlsoiw^'^) = f3w^'^ + w^'^'\ i = iy'-l,...,2,l, 

and with specified nonnegative constrains. Notice that then {B"'v)i ~ n^~^f3"- for i G S"^, 
k>l. 

Let f be a strictly positive vector defined on S\So such that ^|(5\5o,s\So)'^ — ^'^ (take 
for example t'^^-' |5\5o). Define vectors Ual,i = z/', ...2,1, such that Ua]\s\So — and 
""ai Iso = u'^*^ Then 

/:,(z.«) = min (^^'^\s\So,s\So)V + Aks\So,So)f^\ ^ ^j^^^O, ^ = u',...2,l, 
^ Aex\ aiA\^So,s\So)V + A\^So,So)w'^'^ J 

for large enough. Moreover we can additionally choose ctj such that aiXv > ail3v + ai^iv 
for ^ = z/' — 1, . . . 2, 1. Then 

r ^ ( aiXv \_( a,Xv \ ( a^(3v + ai+iv\ _ u) , ^u+i) 

MUaJ > [f^\^^(uJ(^)) J - [pyj(i) + ^^(^+1) J ^ [^yji^ + ^(^+1) J " + ' 

for z = z/' - 1, . . . , 1. It follows that {fJ^{u^ah)i h n^'^P"" for z G 5^, > 1, and the lower 
bound is proved for i in S\So and Sq\Sq. We can now do the same for the states in 5*0. □ 

Corollary 12. The following conditions are equivalent: 

a) Function fx has a strictly positive eigenvector. 

b) Some (every) :<-minimal matrix has a strictly positive eigenvector. 

c) {fxiy))i ~ fof^ ^ ^'^^ Z^'" some (every) vector v > {]. 

d) (/a<:('f^))j ~ U-xi^))] /^'^ ^'i and for some (every) vector w > 0. 
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